Double Mixture and Universal

Ecole normale sup erieure de Paris Given a family of nite dimensional statistical models and a nite number of observations of a random variable, we show how to build a \double mixture" estimator for the density of the random variable whose risk in terms of Kullback-Leibler divergence has a sharp bound compared to the risk of the best model in the family. This estimator is a mixture of model estimators which are themselves mixtures in the continuous parameter spaces of the corresponding models. The idea of using double mixtures has been studied for a long time in the eld of universal compression in coding theory but we highlight the fundamental diierences between our statistical estimator and \twice-universal" coding algorithm, due to the diierence in the criteria to optimize .